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CROSS REFERENCE TO RELATED APPLICATIONS 

This application claims the benefit of U.S. iProvisional Application No. 60/176,108, filed 
January 14, 2000, entitled, "Low-Cost Haptic Mouse Implementations," and this application is a 
continuation-in-part of copending U.S. Patent Applications No. 09/253,132, filed 2/18/99; 
09/456,887, filed 12/7/99; and 09/563,783, filed 5/2/00, which is a continuation of Application No. 
09/103,281, filed 6/23/98, all of which are incorporated by reference herein in their entirety. 



BACKGROUND OF THE INVENTION 

The present invention relates generally to haptic feedback interface devices for use with a 
computer, and more particularly to low-cost haptic devices producing tactile sensations. 

Using an interface device, a user can interact with an environment displayed by a 
computer system to perform fimctions and tasks on the computer, such as playing a game, 
experiencing a simulation or virtual reality environment, using a computer aided design system, 
operating a graphical user interface (GUI), or otherwise influencing events or images depicted on 
the screen. Common human-computer interface devices used for such interaction include a 
joystick, mouse, trackball, steering wheel, stylus, tablet, pressure-sensitive ball, or the like, that is 
comected to the computer system controlling the displayed environment. 

In some interface devices, force feedback or tactile feedback is also provided to the user, 
also known more generally herein as "haptic feedback/' These types of interface devices can 
provide physical sensations which are feh by the user using the controller or manipulating the 
physical object of the interface device. One or more motors or other actuators are used in the 
device and are connected to the controlling computer system. The computer system controls 
forces on the haptic feedback device in conjunction and coordinated with displayed events and 



interactions on the host by sendmg control signals or commands to the haptic feedback device 
and the actuators. 

Many low cost haptic feedback devices provide forces to the user by vibrating the 
manipulandimi and/or the housing of the device that is held by the user. The output of simple 
vibration haptic feedback (tactile sensations) requires less complex hardware components and 
software control over the force-generating elements than does more sophisticated haptic 
feedback. For example, in many current game controllers for game consoles such as the Sony. 
Playstation and the Nintendo 64, one or more motors are mounted in the housing of the controller 
and which are energized to provide the vibration forces. An eccentric mass is positidned on the 
shaft of each motor, and the shaft is rotated unidirectionally to cause the motor and the housing 

■ * 

of the controller to vibrate. The host computer (console unit) provides commands to the 
controller to turn the vibration on or off or to increase or decrease the frequency of the vibration 
by varying the rate of rotation of the motor. 

One problem with these currently-available implementations of haptic feedback devices 
is that the vibrations or other haptic sensations that these implementations produce are very 
limited and caimot be significantly varied, hi addition, gamepad tactile generation devices may 
not be as suitable for other types of interface devices, in particular mouse interfaces or other 
similar position control input devices. The prior art devices also severely limit the haptic 
feedback effects which can be experienced by a user of these devices. 
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SUMMARY OF THE INVENTION 

♦ * 

The present invention is directed to providing low-cost haptic feedback capability to a 
5 mouse interface device and other interface devices that communicate with a host computer or 
controller. The embodiments disclosed herein allow haptic sensations to be output by devices 
that do not require significant design changes to existing interface devices. 

■ 

■ • 

More specifically, in one aspect of the present invention, a haptic feedback mouse device 
for providing haptic sensations to a usier includes a housing physically contacted by the user and 

10 movable in an x-y plane, a sensor coupled to the housing and operative to output a sensor signal 
indicative of the x-y movement, an actuator, and a mass coupled to the actuator, wherein said 
eccentric mass can be rotated by the actuator. The rotation of the mass causes inertial haptic 
sensations to be output on the housing and felt by the user. In one embodiment, the actuator 
rotates the eccentric mass approximately in an x-z plane, a y-z plane, or a combination thereof. 

15 In another embodiment, the actuator rotates the eccentric mass approximately in an x-y plane. 
The inertial force can be a pulse, vibration, or texture correlated with the interaction of a user- 
controlled cursor with a graphical object displayed in a graphical user interface of a host 
computer. 

in another aspect of the present invention, a haptic feedback device includes a housing 
20 physically contacted by the user, where the housing includes a movable portion and a base 
portion, wherein the movable portion is movable with respect to the base portion, and where the 
moveable portion includes a magnet. An actuator is coupled to the housing, and an eccentric 
mass is coupled to the actuator, where the eccentric mass can be rotated by the actuator. 'A 
magnetic interaction between said eccentric mass and said magnet causes an inertial haptic 
25 sensation to be output on said movable portion of said housing and felt by said user when said 
user contacts said movable portion, said inertial haptic sensation influenced by the position of the 
mass. The niovable portion can be a button. The eccentric mass is made of a material that 
interacts magnetically with the magnet, such as iron or steel or a permanently-magnetic material. 

In another aspect of the present invention, a haptic feedback device provides haptic 
30 sensations to a user and includes a housing physically contacted by the user, where the housing 
includes a movable portion and a base portion, where the movable portion is movable with 
respect to the base portion. An actuator is coupled to the housing or to the movable portion, and 
a niass coupled to the actuator, where the mass can be rotated by the actuator. A stop member is 
coupled to the movable portion or the housing and is positioned at least partially in a path of 
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rotation of the mass» where the mass is moved against the stop to produce haptic sensations on 
the movable portion felt by the user contacting the movable portion. The movable portion can be 
a button of the device. Additional stop members can be provided in the range of motion of the 
mass, and inertial and kinesthetic feedback modes can be provided. 

5 In another aspect of the present invention, a haptic feedback mouse device provides 

haptic sensations to a user and includes a device housing physically contacted by the user and 

*...-.•,« 

movable in an x-y plane, where the device housing includes a movable portion and a main 
housing portion, where the movable portion is movable with respect to the main housing portion. 
A moving magnet actuator has an actuator housing coupled to the device housing and a moving 
10 magnet coupled to tihe movable portion, and a sensor outputs a sensor signal indicative of 
housing movement in an x-y plane. In one embodiment, the user can select one of a hierarchy of 
graphical objects by moving the movable portion, wherein a haptic sensation indicates to the user 
a selection of each of the graphical objects in the hierarchy. 

In yet another aspect of the present invention, a haptic feedback mouse device provides 
IS haptic sensations to a user and includes a device housing physically contacted by the user and 
movable in an x-y plane, where the device housing includes a movable portion and a main 
portion. At least part of the movable portion is positioned on a side of the housing and is 
movable with respect to the main portion. A linear actuator has an actuator housing coupled to 
the device housing and an actuated portion coupled to the movable portion, where the linear 
20 actuator moves the movable portion of the device housing linearly away from the main portion of 
the housing when controlled with a control signal, thereby providing a haptic sensation to a user 
contacting the movable portion. A sensor ou^uts a sensor signal indicative of housing 
movement in the x-y plane. Preferably, the movable portion engages a thimib of the user in 
normal operation of the mouse device. 

25 The present invention advantageously provides embodiments for a low-cost h^tic 

feedback device that can output a variety of haptic sensations. The actuators can be implemented 
in existing interface devices with relatively little added expense. The presented features allow 
precision in the control of haptic sensations and a compelling range of sensations to be 
experienced by the user. 

30 These and other advantages of the present invention will become apparent to those skilled 

in the art upon a reading of the following specification of the invention and a study of the several 
figures of the drawing. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIGURE 1 is a perspective view of an interface device system incorporating a haptic 
feedback device present invention; 

FIGURE 2 is a block diagram of a haptic feedback system suitable for use with the 
present invention; 

FIGURE 3 a is a perspective view of a first embodiment of a haptic mouse interface 
device including a eccentric rotating mass providing inertial haptic sensations; 

FIGURE 3b is a perspective view of a second embodiment of a haptic mouse interface 
device including a eccentric rotating mass providing inertial haptic sensations; 

FIGURE 4 is a side elevational view of a haptic mouse interface device including an 
eccentric rotating mass influencing a magnetic button; 

FIGURE 5 is a perspective view of a haptic mouse interface device including an eccentric 
rotating mass engaging a stop member to provide haptic sensations; 

FIGURE 6a is a perspective view of a haptic mouse interface device including a moving 

magnet actuator providing haptic sensations on a button of the device; 

•. ... . • . 

FIGURE 6b is a perspective view of the top and side of the haptic mouse device of Fig. 

6a; ' 

FIGURE 7 is a perspective view of a haptic mouse interface device including a linear 
voice coil actuator providing haptic sensations on a movable housing portion; 

FIGURE 8 is a diagrammatic illustration of a graphical user interface including objects 
associated with haptic sensations; and 

FIGURE 9 is a perspective view of an actuator and transmission for providing forces on a 
button or other movable member. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 



Many of the described embodiments of the present invention add haptic functionality to 
existing mouse designs. Various actuators and assembUes are preferably provided in a mouse 
5 hoiising in ways that do not require significant design and manufacturing changes to the product. 
Mice produced according to these embodiments can fall within the standard mouse price range, 
and these ernbodiments iadd significant new value without forcing the computer user to re-think 
how he or she uses the mouse. 

The below descriptions often refer to a mouse device as a specific embodiment of an 
10 interface device which is suitable for the embodiments of the present invention. However, the 
inventive embodiments described herein are also suitable for a wide variety of other types of 
computer interface devices which can be enhanced with haptic feedback, including trackballs, 
gamepad controllers, joysticks, steering wheels, styhises, touchpads, touchscreehs, Ught guns, 
remote controls, portable computers, knobs, etc. 

15 FIGURE 1 is a perspective view of a haptic feedback mouse interface system 10 of the 

present invention capable of providing input to a host computer and capable of providing haptic 
feedback to the user of the mouse system. Mouse system 10 includes a mouse 12 and a host 
computer 14. It should be noted that the term "mouse" as used herein, indicates an object 
generally shaped to be grasped or contacted from above and moved within a substantially planar 

20 workspace (and additional degrees of freedom if available). 

Mouse 12 is an object that is preferably grasped or gripped and manipulated by a user. 
For example, a user can move mouse 12 to provide planar two-dimensional input to a computer 
system to correspondingly move a computer generated graphical object, such as a curisor or other 
image, in a graphical environment provided by computer 14 or to control a virtual character, 
25 vehicle, or other entity in a game or simulation. In addition, mouse 12 preferably includes one or 
more buttons 16a and 16b to allow the user to provide additional commands to the computer 
system. Each button can typically be pressed down in the degree of fi^eedom of the button for a 
travel distance, at the end of which a button switch is closed and a button signal provided to the 
host computer to indicate the button has been pressed. 

30 Mouse 12 preferably includes one or more actuators 18 which operative to produce tactile 

forces on the mouse housing 12, a portion thereof, and/or a button 16. This , operation, is 
described in greater detail below with reference to Figs. 3a-7. 
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Mouse 12 rests on a ground surface 22 such as a t^bletop or mousepad. A user grasps the 
mouse 12 and moves the mouse in a planar workspace on the surface 22 as indicated by arrows 
24. Mouse 12 may be moved anywhere on the ground surface 22, picked up and placed in a 
different location, etc. A fiictional ball and roller assembly (not shown) can in some 
embodiments be provided on the imderside of the mouse 12 to translate the planar motion of the 
mouse 12 into electrical position signals, which are sent to a host computer 14 over a bus 20 as is 
well known to those skilled in the art. In other embodiments, different mechanisms and/or 
electronics can be used to convert mouse motion to position or motion signals received by the 
host computer. For example, optical sensors can be used; a suitable optical mouse technology is 
made by Hewlett Packard of Palo Alto, CA, where both the optical emitter and detector are 

4 

provided on the mouse housing and detect motion of the mouse relative to the planar support 
surface by optically taking and storing a number of images of the surface and comparing those 
images over time to determine if the mouse has moved. Altematively, a portion of an optical 
sensor can be built into the surface 22 to detect the position of an emitter or transmitter in mouse 
12 and thus detect the position of the mouse 12 on the surface 22. Mouse 12 is preferably a 
relative device, in which its sensor detect a change in position of the mouse, allowing the mouse 
to be moved over any surface at any location. An absolute mouse may also be used, in which the 
absolute position of the mouse is known but the mouse is typically limited to a particular 
predefined workspace. 

Mouse 12 is coupled to the computer 14 by a bus 20, which communicates signals 
between mouse 12 and computer 14 and may also, in some preferred embodiments, provide 
power to the mouse 12. Components such as actuator 18 require power that can be suppli^ from 
a conventional serial port or through an interface such as a USB or Firewire bus. In other 
embodiments, signals can be sent between mouse 12 and computer 14 by wireless 
transmission/reception. In some embodiments, the power for the actuator can be supplemented 
or solely supplied by a powder storage device provided on the mouse, such as a capacitor or one or 
more batteries. Some embodiments of such are disclosed in Patent No. 5,691,898, incorporated 
herein by reference. 

Host computer 14 can be a personal computer or workstation, such as a PC compatible 
computer or Macintosh personal computer, or a Sun or Silicon Graphics workstation. For 
example, the computer 14 can operate under the Windows™ , MacOS, Unix, or MS-DOS 
operating system. Altematively, host computer system 14 can be one of a variety of home video 
game console systems commonly connected to a television set or other display, such as systems 
available from Nintendo, Sega, or Sony. In other embodiments, host computer system 14 can be 
a "set top box" which can be used, for example, to provide interactive television frmctions to 
users, or a "network-" or "internet-computer" which allows users to interact with a local or. 
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global network using standard connections and protocols such as used for the Internet and World 
Wide Web. Host computer preferably includes a host microprocessor, random access memory 
(RAM), read only memory (ROM), input/output (I/O) circuitry, and other components of 
computers well-known to those skilled in the art. 

S Host computer 14 preferably implements a.host application program with which a user is 

interacting via mouse 12 and other peripherals, if appropriate, and which may include force 
feedback functionality. For example, the host application program can be a video game, word 
processor or spreadsheet, Web page or browser that implements HTML or VRML instructions, 
scientific analysis program, virtual reality training program or application, or other application 

10 program that utilizes input of mouse 12 and outputs force feedback commands to the mouse 12. 
Herein, for simplicity, operating systems such as Windows™, MS-DOS, MacOS, Linux, Be, etc. 
are also referred to as "application programs." In one preferred embodiment^ an application 
program utilizes a graphical user interface (GUI) to present options to a user and receive input 
firom the user. Herein, computer 14 may be referred as providing a "graphical environment,", 

15 which can be a graphical user interface, game, simulation, or other visual environment. The 
computer displays "graphical objects" or "computer objects," which are not physical objects, but 
are logical software imit collections of data and/or procedures that may be displayed as images "by 
computer 14 on display screen 26, as is well known to those skilled in the art. A displayed cursor 
or a simulated cockpit of an aircraft might be considered a graphical object The host application 

20 program checks for input signals received from the electronics and sensors of mouse 12, and 
ou^uts force values and/or commands to be converted into forces ou^ut for mouse 12. Suitable 
software drivers which interface such simulation soft^vare with computer input/output (I/O) 
devices are available from Immersion Corporation of San Jose, CaUfomia. 

Display device 26 can be included in host computer 14 and can. be a standard display 
25 screen (LCD, CRT, flat panel, etc.), 3-D goggles, or any other visual ouQ)ut device. Typically, 
the host apphcation provides images to be displayed on display device 26 and/or other feedback, 
such as auditory signals. For example, display screen 26 can display images from a GUI. 

In altemative embodiments, the mouse can be a different interface or control device. For 
example, a hand-held remote control device used to select fimctions of a television, video 
30 cassette recorder, sound stereo, intemet or network computer (e.g., Web-TV™), or a gamepad 
controller for video games or computer games, can be used with the haptic feedback components 
described herein. 

FIGURE 2 is a block diagram illustrating one embodiment of the force feedback system 
suitable for use with any of the described embodiments of the present invention and including a 
35 . local microprocessor and a host computer system. 
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Host computer system 14 preferably includes a host microprocessor 100, a clock 102, a 
display screen 26, and an audio output device 104. The host computer also includes other well 
known components, such as random access memory (RAM), read-only memory (ROM), and 
input/ou^ut (I/O) electronics (not shown). Display screen 26 displays images of a game 
5 . environment, operating system application, simulation, etc. Audio output device 104, such as 
speakers, is preferably coupled to host microprocessor 100 via ampUfiers, filters, and other 
circxiitry well known to those skilled in the art and provides sound output to user when an "audio 
event" occurs during the implementation of the host application program. Other types .of 
peripherals can also be coupled to host processor 1 00, such as storage devices (hard disk drive, 
10 CD ROM drive, floppy disk drive, etc.), printers, and other input and output devices. 

Mouse 12 is coupled to host computer system 14 by a bi-directional bus 20 The bi- 
directional bus sends signals in either direction between host computer system 14 and the 
interface device. Bus 20 can be a serial interface bus, such as an RS232 serial interface, RS-422, 
Universal Serial Bus (USB), MIDI, or other protocols well known to those skilled in the art; or a 
15 parallel bus or wireless link. For example, the USB standard provides a relatively high speed 
interface that can also provide power to actuator 18. 

Mouse 12 can include a local microprocessor 110. Local microprocessor 110 can 
optionally be included within the housmg of mouse 12 to aUow efficient communication with 
other components of the mouse. Processor 110 is considered local to mouse 12, where "local" 

20 herein refers to processor 110 being a separate microprocessor from any processors in host 
computer system 14. "Local" also preferably refers to processor 110 being dedicated to haptic 
feedback and sensor I/O of mouse 12. Microprocessor 110 can be provided with software 
instmctions (e.g., firmware) to wait for commands or requests fcom computer host 14, decode the 
command or request, and handle/control input and output signals according to the command or 

25 request. In addition, processor 110 can operate independently of host computer 14 by reading 
sensor signals and calculating appropriate forces from those sensor signals, time signals, and 
stored or relayed instructions selected in accordance with a host command. Suitable 
microprocessors for use as local microprocessor 110 include the MC68HC711E9 by Motorola, 
the PIC16C74 by Microchip, and the 82930AX by Intel Corp., for example, as well as more 

30 sophisticated force feedback processors such as the Immersion Touchsense Processor. 
Microprocessor 110 can include one microprocessor chip, multiple processors and/or co- 
processor chips, and/or digital signal processor (DSP) capability. 

Microprocessor 110 can receive signals from sensor 112 and provide signals to actuator 
18 in accordance with instructions provided by host computer 14 over bus 20. For example, in a 
35 ■ local control embodiment, host computer 14 provides high level supervisory commands to 

■ 
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microprocessor 110 over bus 20, and microprocessor 110 decodes the comimands and manages 
low level force control loops to sensors and the actuator in accordance with the high level 
commands and independently of the host computer 14, This operation is described in greater 
detail in US Patents 5,739,81 1 and 5,734,373, both incorporated by reference herein. In the host 
5 control loop, force commands are output from the host computer to microprocessor 110 and 
instruct the microprocessor to output a force or force sensation having specified characteristics. 
The local microprocessor 110 reports data to the bost computer, such as locative data that 
describes the position of the mouse in one or more provided degrees of freedom. The data can 
also describe the states of buttons 16 and safety switch 132. The host computer uses the locative 

10 data to update executed programs. In the local control loop, actuator signals are provided from 
the microprocessor 110 to actuator 18 and sensor signals are provided from the sensor 112 and 
other input devices 118 to the microprocessor 110. Herein, the term "tactile sensation*' refers to 
either a single force or a sequence of forces output by the actuator 1 8 which provide a sensation 
to the user. For example, vibrations, a single jolt, or a texture sensation are all cbnsidered tactile 

15 sensations. The microprocessor 110 can process inputted sensor signals to determine appropriate 
output actuator signals by following stored instructions. The microprocessor may use sensor 
signals in the local determination of forces to be output on the user object, as well as reporting 
locative data derived from the sensor signals to the host computer. 

In yet otiier embodiments, other hardware can be provided locally to mouse 12 to provide 
20 fimctionality similar to microprocessor 110. For example, a hardware state machine 
incorporating fixed logic can be used to provide signals to the actuator 18 and receive sensor 
signals from sensors 1 12, and to output tactile signals according to a predefined sequence, 
algorithm, or process. Techniques for implementing logic with desired fimctions in hardware are 
well known to those skilled in the art. Such hardware can be better suited to less complex force 
25 feedback devices, such as the device of the present invention. 

In a different, host-controlled embodiment, host computer 14 can provide low-level force 
commands over bus 20, which are directly transmitted to the actuator 18 via microprocessor 110 

■ • 

or other circuitry. Host computer 14 thus directly controls and processes all signals to and from 
the mouse 12, e.g. the host computer directly controls the forces output by actuator 18 and 

30 directly receives sensor signals from sensor 112 and input devices 118. This embodiment may be 
desirable to reduce the cost of the force feedback device yet ftirther, since no complex local 
microprocessor 110 or other processing circuitry need be included in the mouse. Furthermore, 
since one actuator 18 is used with forces not.provided in the primary sensed degrees of freedom, 
the local control of forces by microprocessor 110 may not be necessary in the present invention 

35 to provide the desired quality of forces. Other embodiments may employ a "hybrid" organization 
where some types of force effects (e.g. closed loop effects or high frequency effects) are 

• * 
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controlled purely by the local microprocessor, >yhile other types of effects (e.g., open loop or low 
frequency effects) may be controlled by the host. 

In the simplest host control embodiment, the signal from the host to the device can be a 
single bit that indicates whether to pulse the actuator at a predefined frequency and magnitude. 
S In a more complex embodiment, the signal from the host could include a magnitude, giving the 
strength of the desired pulse. In yet a more complex embodiment, the signal* can* include a 
direction, giving both a magnitude and a sense for the pulse. In still a more complex 

embodiment, a local processor can be used to receive a simple command from the host that 

.• • • 

indicates a desired force value to apply over time. The microprocessor then outputs the force 
10 value for the specified time period based on the one command, thereby reducing the 
communication load that must pass between host and device. In an even more complex 
embodiment, a high-level command with tactile sensation parameters can be passed to the lodal 
processor on the device which can then apply the fiiU sensation independent of host intervention. 
Such an embodiment allows for the greatest reduction of communication load. FihaUy, a 
IS combination of numerous methods described above can be used for a single mouse device 12. 

Local memory 122, such as RAM and/or ROM, is preferably coupled to microproceiisor 
110 in mouse 12 to store instructions for microprocessor 110 and store temporary and other data. 
For example, force profiles can be. stored in memory 122, such as a sequence of stored force 
values that can be output by the microprocessor, or a look-up table offeree values to be output 

20 based on the ciirrent position of the user object. In addition, a local clock 124 can be coupled to 
the microprocessor 110 to provide timing data, similar to system clock 18 of host computer . 12; 
the timing data might be required, for example, to compute forces output by actuator 1 8 (e.g., 
forces dependent on calculated velocities or other time dependent factors). In embodiments 
using the USB commimication interface, tindng data for microprocessor 110 can be alternatively 

25 retrieved from the USB signal. 

In some embodiments, host computer 14 can send a "spatial representatioii'* to the local 
microprocessor 110, which is data describing the locations of some or all the graphical objects 
displayed in a GUI or other graphical environinent which are associated with forces and the 
characteristics of these graphical objects. The microprocessor can store such a spatial 
30 representation in local memory 122, and thus will be able to determine interactions between the 
user object and graphical objects (such as the rigid surface) independently of the host compute. 
Also, the local memory can store predetermined force sensations for the microprocessor that are 
to be associated with particular types of graphical objects. 

« 

Sensors 1 12 sense the position or motion of the mouse device (e.g. the housing 50) in its 
35 planar degrees of freedom and provides signals to microprocessor 110 (or host 14) including 
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infonnation representative of the position or motion. Sensors suitable for detecting planar 
motion of a mouse include digital optical encoders fictionally coupled to a rotating ball or 
cylinder, as is well known to those skilled in the art. Optical sensor systems, linear optical 
encoders, potentiometers, optical sensors, velocity sensors, acceleration sensors, strain gauge, or 
5 other types of sensors can also be used, and either relative or absolute sensors can be provided. 
Optional sensor interface 114 can be used to convert sensor signals to signals that can be 
interpreted by the microprocessor 110 and/or host computer system 14, as is well known to those 
skilled in the art. 

Actuator(s) 18 transmits forces to the housing 50, button 16, or other portion of the 

10 mouse in response to signals received from microprocessor 110 and/or host computer 14, and is 
described in greater detail below. Many types of actuators can be used, including a rotary DC 
motors, voice coil aqtuators, moving magnet actuators, pneimiatic/hydraulic actuators, solenoids, 
speaker voice coils, piezoelectric actuators, passive actuators (brakes), etc. In many of the 
implementations herein, the actuator has the ability to apply short duration force sensation on the 

15 housing or handle of the mouse. This short duration force sensation is described herein as a 
"pulse." The "pulse" can be directed substantially along a Z axis orthogonal to the X-Y plane of 
motion of the mouse. In progressively more advanced embodiments, the magnitude of the 
"pulse" can be controlled; the sense of the "pulse" can be controlled, either positive or negative 
biased; a *^eriodic force sensation" can be applied on the handle of the mouse, where the 

20 periodic sensation can have a magnitude and a frequency, e,g. a sine wave; the periodic sensation 
can be selectable among a sine wave, square wave, saw-toothed-up wave, saw-toothed-down, and 
triangle wave; an envelope can be appUed to the period signal, allowing for variation -in 
magnitude over time; and the resulting force signal can be "impulse wave shaped" as described 
in U.S. Patent No. 5,959,613. There are two ways the period sensations can be communicated 

25 from the host to the device. The wave forms can be "streamed" as described in U.S. Patent 

5,959,613 and provisional patent appUcation 60/160,401, both incorporated herein by reference 

I ■ - 

in their entirety. Or the waveforms can be conveyed through high level commands that include 
parameters such as magnitude, frequency, and duration, as described in U.S. Patent 5,734,373. 

Alternate embodiments can employ additional actuators for providing tactile sensations or 

r 

30 forces in the planar degrees of freedom of the mouse 12. For example, the mouse can be 
enhanced with a secondary actuator. Because of power constraints, this secondary means can be 
passive (i.e., it dissipates energy) in some embodiments. The passive actuator can be a brake, 
such as a magneto-rheological fluid brake or magnetic brake. The passive braking means can be 
employed through a frictional coupling between the mouse housing and the table surface 22. 

35 When the brake is engaged, the user can feel the passive resistance to motion of the mouse (in 
one or two degrees of freedom). Actuator interface 116 can be optionally connected between 



Docket No. IMM1P098A 



12 



actuator 18 and microprocessor 110 to convert signals from microprocessor 110 into signals 
appropriate to driye actuator 18. Interface 38 can include power amplifiers, switches, digital to 
analog controllers (DACs), analog to digital controllers (ADCs), and other . components, as is 
well known to those skilled in the art. 

Other input devices 118 are included in mouse 12 and send input signals to 
microprocessor 110 or to host 14 when manipulated by the user. Such input devices include 
buttons 16 and can include additional buttons, dials, switches, scroll wheels, or other controls or 
mechanisms. 

• - ■ 

Power supply 120 can optionally be included in mouse 12 coupled to actuator interface 
116 and/or actuator 18 to provide electrical power to the actuator, or be provided £is a separate 
component. Alternatively, and more preferably, power can be drawn from a power supply 
separate from mouse 12, or power can be received across a USB or other bus. Also, received 
power can be stored and regulated by mouse 12 and thus used when needed to drive actuator 18 
or used in a supplementary fashion, as described in copending application no. 09/456,887, filed 
12/7/99, and incorporated herein by reference in its entirety. A safety switch 132 can optionally 
be included to allow a user to deactivate actuator 18 for safety reasons. 



Embodiments of the Present Invention 

Several embodiments of mouse interface device 12 providing haptic sensations to the user 
are described below. Preferred embodiments provide one or more of several desirable 
characteristics fqr a haptic mouse designed for the consumer market. One desirable characteristic 
is that the mouse should feel like it is "aUve" to the user, like the forces are coupling into the 
user's body. The "alive" quality is often determined by system compUance, actuator authority, 
and transmissibility into the hand. Furthermore, it is preferred that the moving member or 
portion be spring centered so that vibrations / forces do not disappear or get cUpped. Preferably, 
user effort is not reqmred to maintain contact with the moving feedback surface while using the 
mouse. The mouse preferably also provides feedback for a range of user grip postures, e.g. 
palming, gripping, and finger tip iisage. If possible, the haptic feedback should be in an axis that 
is substantially de-coupled from position input in the x-y plane. Preferably, the haptic feedback 
does not interfere with button operation by the user or button closure perception, and the mouse 
should work seamlessly as a normal mouse when the user is not paying attention to forces. The 
mouse should have very good fideUty at high frequencies (e.g., 200 to 20 Hz) and convey lower 
frequencies (e.g., <20 Hz) with enough displacement that they are perceptible. Overall, the 
haptic mouse should add value with minimal sacrifice and cost. 

. •••••• . 
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FIGURE 3a is a perspective view of a mouse device 200 providing tactile sensations to a 
user with an eccentric rotating mass to provide inertial forces, such as vibrations. A lower base 
portion 202 of the mouse housing can include a ball sensor 204, a mouse \yheel 206, circuits (not 
shown), and other standard components. In addition, a rotary motor 208 can be coupled to the 
5 base 202, where a rotary shaft 210 of the motor is coupled to an eccentric mass 212 positioned so 
that the center of mass of the mass 212 is offset from the center of rotation of the shaft 210. A 
cover portion 214, shown in dashed lines, can be normally positioned over the base portion 202. , 

« 

The eccentric mass 212 is rotated by the motor 208 to cause inertial tactile sensations on 
the mouse housing. The inertial sensations are caused by the inertia produced by the eccentric 

10 rotation of the mass, which causes a wobbling motion that is transmitted through actuator to the 
housing. The user contacting the housing can feel the sensations. The sensations can be 
determined &om host commands, signals, or local determination, as explained above. In one 
ernbodiment, the mass 212 is rotated in a single direction. In another embodiment, the mass 212 
can be rotated harmonically (in two directions). Some mouse embodiments can allow both imi- 

15 directional and bi-directional modes, e.g. a host command from the host computer can determine 
which mode is currently operational. 

In embodiment 200, the motor 208 is positioned such that the eccentric mass 212 rotates 
in approximately the y-z plane, where the shaft of the motor extends parallel to the x-axis. Thus, 
the inertial forces output by the rotation of the mass are along the y- and z-axes. If the mass is 

20 rotated quickly enough and/or if the inertial forces on the housing are of high enough magnitude, 
the mouse may be moved or vibrated along the y-axis and the portion of the forces output in the 
y-axis may cause a controlled object, such as a displayed cursor, to change its y position in a 
graphical environment in response to motor activation. If this effect is undesired, it can be 
alleviated in some embodiments by providing a selective disturbance filter,, as described in U.S. 

25 Patent No. 6,020,876 and incorporated herein by reference in its entirety. 

The embodiment 200. can produce strong forces to the user if the mass 212 is rotated 
quickly. In some embodiments, forces ou^ut to the user can be dependeint on the. initial state. of 
the motor/mass. For example, if the eccentric mass were initially positioned at the bottom of its 
rotational range, a "pop'' sensation (e.g. one or a small number of quick mass rotations) would 

30 . feel different than if the mass were initially positioned at the top ' of its range. Rotating mass 
control firmware and a sensor that reads mass rotational position may be used to improve the 
eccentric mass control and make particular force sensations always feel the same. For example, . 
copending application 09/669,029, filed . 9/25/00, describes methods to . control an eccentric 
rotating mass that can be used in the present invention, and is incorporated herein by reference in 

35 its entirety. A harmonic drive, in which the mass is driven in both directions about its rotational 
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axis, higher-fidelity force effects may, in' general, be obtained, as described in copending 
application 09/608,125, which is incorporated herein by reference in its entirety. Also, firmware 
or control software can be used to translate low frequency periodic drive signals into short 
duration pulses that start the mass moving from a known position. 

In some embodiments, the eccentric mass 212 can be driven harmonically (bi- 
directionally) against one or more stop members, such as pins, that are coupled to the base 202 or 
cover 214 of the mouse housing. The impact force of the mass against the stop members causes 
different types of force sensations that can be provided instead of or in addition to inertial 
sensations. Sensations resulting from such stop members is described in greater detail below. 

FIGURE 3b is a perspective view of a mouse device 220 providing tactile sensations to a 
user with an eccentric rotating mass. Embodiment 220 is similar to mouse 200 described above, 
and can include a lower base portion 222, a ball (or other type) senisor 224, a mouse wheel 226, 
circuits (not shown), and other standard components. A rotary motor 228 can be coupled to the 
base 222, where a rotary shaft 230 of the motor is coupled to an eccentric mass 232 positioned so 
that the center of mass of the mass 232 is offset from the center of rotation of the shaft 230. A 
cover portion 234, shown in daished lines, can be normally positioned over the base portion 222. 

Embodiment 220 differs from embodiment 200 in that the motor 228| is positioned such 
that the shaft 230 is parallel to the z-axis and rotates the eccentric mass 232 in the x-y plane. The 
inertial sensations are similar to those produced by embodiment 220, except that the forces are 
provided in the x-y plane. If the inertial sensations are low enough magnitude^ then targeting 
activities of the mouse are typically unaffected. If the inertial sensations are strong enough, 
however, they may cause the mpuse and any controlled graphical object to be moved in the x-y 
plane, possibly throwing ofF the cursor from a desired target, and thus may be more undesirable 
than the embodiment 200 which only may cause mouse movement along the y-axis. Smaller 
masses 232 (and thus smaller forces) can reduce the disturbances. This embodiment may be 
suitable as an "anti-targeting" device; e.g. a particular game or other application may require or 
desire forces that prevent a user from targeting a cursor or other object accurately. The other 
features described for enibodiment 200 can also be employed for embodiment 220. 

FIGURE 4 is a side elevational view of another embodiment 250 of a tactile mouse which 
can output haptic sensations on a mouse button or other. moveable portion of an interface device. 
. Mouse 250 can include the standard device components detailed above. Mouse 250 includes a 
motor 252 coupled to the housing of the mouse, such as a DC rotary (e.g. pager) motor or other 
type of actuator, and which rotates an eccentric mass 254. For example, the motor 252 is 
mounted to the bottom 253 of the mouse housing 251 in the embodiment shown. The mass can 
be rotated in any configuration, but the rotating motor shaft is preferably oriented in the x-y plane 
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so that the eccentric mass 254 rotates in a y-z plane or an x-z plane, or a combination of both. 
Mouse 250 also includes a button 256 to which a permanent magnet 258 is coupled. In the 
embodiment shown, the magnet 258 is coupled to the xmderside of the button 256. Button 256 is 
hinged and can move approximately as shown by arrow 260. The user can depress the button to 
activate a switch and send a button signal to the host computer, as is well knpwn on mouse and 
other interface devices. ' 

The eccentric mass 254 can be controlled similarly to the eccentric masses described 
above to provide inertial tactile sensations to the user contacting the housing of the mouse. For 
example, the mass 254 can be rotated in one direction or can be controlled harmonically to move 
in two directions about its . rotational axis to provide the desired inertial sensations. The 
harmonic control tends to more efficiently couple vibrations to the housing inertially at higher 
frequencies. 

Furthermore, embodiment 250 allows tactile sensations to be output on the button 256. 
When the eccentric mass 254 is rotated to the top of its rotational range, i.e., itis closest position 
to the magnet 258, the mass magnetically influences the button 256 by attracting the magnet 258 
toward the mass 254. For example, the mass 254 can be made of a metal, such as iron or steel, 
that magnetically interacts with the magnet 258. If the mag;netic attraction force is strong 
enough, it may cause the button 256 to move in the direction toward the mass 254; however, the 
forces are preferably made sufficiently weak to not cause the button switch to close. This allows 
the user to press the button when desired with little or no interference from forces output in the 
button's degree of freedom. For example, the button travel range can be made large enough and 
can include a sensor to detect button position, so that when the button reaches a position near to 
the button switch, the forces are reduced by moving the mass away, allowing a button click 
uninfluenced by the magnetic forces. 

As the mass 254 rotates away from the magnet 258, the magnetic attraction force reduces 
in magnitude, and the button 256 is allowed to move back to its origin position due to a physical 
centering spring provided on the button 256 (e.g., the centering spring can be provided within the 
hinge of the button, or . is a separate physical spring). Thus, the button 256 experiences an 
oscillating magnetic force (e.g., a vibration) if the mass 254 is continually rotated in one 
direction, where the frequency of oscillation is controlled by the frequency of rotation of the 
mass. If the user is contacting the button, the user experiences haptic sensations through the 
button; these sensations may include actual motion of the button up or down in the degree of 
freedom of the button. The user also may experience inertial tactile sensations through the 
housing of the mouse caused by the rotation of the eccentric mass. 
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Alternatively, the motor 252 and eccentric mass 254 can be used to impart forces in the 
degree of freedom of the button 256 in a "kinesthetic button mode/' Jn this mode, kinesthetic 
forces such as resistance to movement of the button in its degree of freedom, spring forces in the 
button degree of freedom, damping forces in the button degree of freedom, etc., can be output. A 
5 particular magnitude of the kinesthetic force is determined by the position of the mass with 
respect to the magnet at that point in time. Thus, a strong attraction (or resistive) force is applied 
. when the mass is very close to the magnet, while a weaker attraction (or resistance) is applied 
when the mass has been rotated to a position further from the magnet. Mass position can be 
modulated according to the desired relationship, e.g. a spring force is created by providing a 
10 resistive force having a magnitude based on the current position of the button 256 in its degree of 
freedom (the current button position can be read by a dedicated sensor). A mapping of eccentric 
mass position to resistance (or attractive force) magnitude can be provided, e.g. the local 
microprocessor can access such a mapping to determine how to control mass position. 

If the eccentric mass is made of a metal such as iron or steel, the force between magnet 
15 and mass are attractive. In other embodiments, the mass 254 can be made of a permanent 
magnetic material. Depending on the polarities of the sides of the magnet 258 and mass 254 
facing each other, the magnetic force will then either be attractive or repulsive, allowing either an 
attractive or repulsive force on the button 256, Li some embodiments, both attractive and 
repulsive forces can be implemented, and either can be selected by the local microprocessor, host 
20 computer, etc. For example, if flux is added or subtracted from a steel or iron mass 254, 
attractive or repulsive forces can be implemented. For example, a wire coil can be wrapped 
around the mass 254 and a current flowed therethrough (the cmrent can be controlled by a local 
processor, for example), allowing flux to be added or subtracted and thus allowing both attractive 
and repulsive forces to be implemented. 

25 In some embodiments, the mass can also be rotated bi-directionally using harmonic 

control, as described above. For example, a sine wave can control the harmonic motion of the 
mass, allowing vibrations to be imparted on the button 256. 

The inouse can also be provided with multiple different modes, each mode moving the 
mass ia a different way or according to a different control method to produce a different type of 

30 haptic sensation. For example, firmware on the mouse processor, and/or host software, can 
selectively control this multiple-mode ability. For example, tactile and kinesthetic modes can be 
provided. In one example, when the cursor is moved within a displayed window, a vibration can 
be output on the button 256 in tactile mode. When the user presses the button to select an icon in 
that window, kinesthetic mode can be initiated and a spring force can be output on the button to 

35 resist the button's motion downward (or attract the button to decrease the force necessary for the 
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user to push the button). Other embodiments can also or alternatively include harmonic and imi- 
directional mass rotation modes for different types of tactile sensations. 

Multiple buttons of the mouse or other interface device can include a magnet 258. Each 
button can have an eccentric motor/mass dedicated to that button, or multiple buttons can be 
5 magnetically influenced by a single motor and/or eccentric mass. In yet other embodiments, 
other moving portions of the mouse 250 can be provided with a magnet similar to magnet 258 
and be moved with respect to the '"base portion" of the mouse, which in this embodiment is the 
remaroing portion of the housing except the movable portion. For example, a cover portion of 
the mouse hinged to the base portion can be provided with a magnet so that the entire cover 
10 portion is vibrated or induced with magnetic forces based on the position of the eccentric mass 
254 during its rotation. Or, a portion of the housing tiiat is pivotally or translatably coupled to 
the rest of the housing can be magnetically influenced. Some embodiments of moveable mouse 
portions are described in U.S. Patent 6,088,019, incorporated herein by reference in its entirety. 

FIGURE 5 is a perspective View of another embodiment 270 of a mouse providing haptic 
15 sensations on a button. The upper portion 272 of mouse 270 is shown, which is intended to mate 
with a bottom portion, e.g. a base similar to those shown with respect to Figs. 3a and 3b, or other 
type of base. Two mouse buttons 274 and 276 are shown from the underside of the upper portion 
272. The buttons 274 and 276 are coupled to the housing portion 272 at a hinge 278. The 
housing of a rotary motor 280 is coupled directiy to the button 276 such that the button 276 can 
20 still be moved and pressed by the user in normal fashion; when the button is moved, the motor 
280 is also moved. An eccentric mass 282 is coupled to a rotating shaft 284 of the motor 280. 
The mass 282 can be similar to the eccentric masses described above. 

■ • « 

A number of eccentric rotating mass motors, voice-coils, speaker actuators, and/or other 
types of actuators can be attached to a displaceable surface of the mouse, such as the mouse 

25 button 276 or a moveable portion of the top or side of the mouse housing, for example. These 
actuators can all produce a vibration on the displaceable surface. Thus, a freely-rotating mass 
282 will produce a vibration on the button 276 to which the motor 280 is attached due to the 
inertial forces. Some actuators are capable of harmonic drive, providing high bandwidth at the 
expense of power consumption.. Harmonically-driven actuators are able to produce vibrations as 

30 wellas**cUcks", e.g. single pulses offeree. 

In other embodiments, an grounded stop 284 can be positioned in the rotatable range of 
the mass 282 to block the rotation of the mass. For example, the stop 284 can be a pin or screw 
that is moimted to the housing 272 and extends into the rotational range of the mass. In 
unidirectional operation, a force can be applied to the button 276 by driving the mass 282 against 
35 the stop 284. Since the stop 284 is grounded, this causes the motor 280 and button 276 to move 
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in the degree of freedom of the button as the mass 282 pushes against the stop 284. In some 
embodiments, the resixlting force may not be of sufficient magnitude to actually move the button 
and motor, but a force is appUed to the motor and button in the button's degree of freedom. 

Alternatively, the actuator 280 can be grounded to the housing 272 while the stop 284 is 
5 coupled to the movable portion, such as button 276. This can provide similar sensations to those 
generated by a grounded stop and floating actuator. 

Similar to the embodiment of Fig. 4, different tactile modes can be provided; in some 
embodiments, one of multiple modes can be selected by the controller of the motor 280. For 
example, in a vibration mode, a series of discrete activation pulses can be sent to the motor 280 
10 to drive the eccentric mass 282 against the stop 284 at regular periodic (or irregular, if desired) 
intervals, causing a vibration on the button. 

Kinesthetic forces for a kinesthetic mode are not easily achieved except for the 
embodiments where an actuator engages one or more limiting stops 284 and can then displace the 
movable surface if current is controlled. For example, in a kinesthetic force nlode, the mass 282 
15 can be driven continuously against the stop 284 to cause a constant resistance force on the buttpn 
276 in its degree of freedom, or other type offeree. For example, a spring force can be output by 
controlling the constant force on the button to be dependent on button position according to the 
relation F — kx, where x is the position of the button in the button's degree of freedom (a 
dedicated sensor can be provided to detect button position in the button degree of freedom). 

20 In harmonic operation, the mass 282 can be driven in two directions, so that the mass can 

provide a vibration when it is between stops, and can be impacted with the stop 284 on either 
side of the stop to provide kinesthetic sensations or a different type of vibration sensation. For 
example, a variety of vibration sensations can be provided, such as moving the mass against 
either side of a stop altemately, or by driving the mass against the stop, then moving it away, etc. 

25 A kinesthetic mode can be controlled in either direction of the button in its degree of freedom by 
moving the mass against a corresponding side of the stop and causing a force on the button by 
continuously forcing the mass against the stop. In some embodiments, two stops can be provided 
to define a range of rotation for the mass 282. Such a configuration can cause a vibration on the 
button when the mass is operated harmonically between limit stops, and can provide a kinesthetic 

30 force control mode when the mass is forced against one of the stops. Actuators such as a spring 
biased solenoid can also be used since these actuators can be harmonic or can provide two basic 
forces from impact if driven to the end of their stroke. 

■ 

Other embodiments described herein, such as those of Figs. 3a and 3b, can also employ . 
one or more stops in the range of motion of the eccentric mass to provide different haptic 
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sensations. Another example of a tactile mouse includes an eccentric rotating mass motor 
coupled to mouse housing or the movable portion, and two. stop members coupled to the other of 
the movable portion or mouse housing. The stop members defining a range of rotation of the 
mass. The rotating mass can shake the mouse housing and transmit ihertial vibrations when 
operated harmonically between the limits defined by the stops. Then, if the motor is brought to 
bear against one of the stop members, the button surface may be displaced by controlling the 
motor current. This kind of motor working against a stop member is not like a bi-directional 
linear actuator because there is an inherent dead band, but spring effects can still be output in one 
direction of the button or the mass can intentionally impact the stop to generate "pops." 

Some embodiments of mouse 270 may have incoiisistent force output for reasons similar 
to other eccentric rotating mass embodiments: the initial conditions (position and velocity) of the 
eccentric mass may influence how the actuator operates in response to different drive input 
signals. As a result, the force effects may not feel repeatable or consistent and may be 
undesirable. For example, a command signal that commands a pulse effect when the cursor 
crosses over an icon may cause the force effect to be output too late, after the icon was crossed by 
the cursor, due to the time it takes for the mass to be accelerated against a stop. Li some cases, 
rebound forces may counteract the next pulse and obscure subsequrat effects. Such 
disadvantages may be solved in some embodiments by providing controlling methods and/or a 
sensor that detects mass rotational position that maintain the mass in a known position so that 
force sensations are repeatable and consistent. Gamepad motor control as described in 
application no. 09/669,029 may also be used. 

FIGURES 6a and 6b are perspective views of another embodinient 300 of a tactile mouse 
of the present invention. Jn Fig. 6a, an upper portion 302 of mouse 300 is shown, which is 
intended to mate with a bottom portion, e.g. a base similar to those shown with respect to Figs. 
3a and 3b, or other type of base. Two mouse buttons 304 and 306 are shown from the underside 
of the upper portion 302. 

In embodiment 300, a moving-magnet actuator 310 is grounded to the housing 302. A 
moving magnetic portion 311 and bearing of the actuator 310 rotates about axis A and is coupled 
to the mouse button 306 by an extension member 313 which is guided by a support structure 312. 
Thus, the rotation of the moving magnet causes a force on the button 306 about that axis and 
directly in the degree of freedom of the button, allowing forces in either direction of that button*s 
degree of freedom to be output when rotary forces are output by the actuator. This causes the 
button to pivot approximately about the axis of rotation. This motion of button 306 is shown in. 
Fig. 6b by arrow 314. For example, half of a moving-magnet actuator as described in copending 
apphcation no. 09/565,207, incorporated herein by reference in its entirety, can be used for 



1 
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actuator 310. Other types of moving-magnet actuators can also be used. In one embodiment, the 
actuator can produce several ounces of force at the button leading edge (the front tip of the 
button) where the stroke is, for example, about +/- 0.125 in. The direct drive moving magnet 
implementation is capable of very high fidelity haptics. The buttons 304 and 306 can be coupled 
5 to the housing portion 302 at a hinge 308, or may be coupled only to the moving magnetic 
portion 3 1 1 or shaft of the actuator. 

This embodiment can also be realized w^ith a number of actuators and transmissions. 
Other embodiments and features of providing haptic feedback on a mouse button or other types 
of buttons are described in copending application nos. 09/253,132 and 09/156,802, both 
10 incorporated herein by reference in their entirety. The forces are output approximately along the 
z-axis since the button moves approximately along that axis, and therefore the forces need not 
interfere with the movement of the mouse in the x-y plane. This makes it also well suited to 
providing the feel of a third dimension in relation to the two-dimensional plane of a display 
screen. 

15 In some embodiments, the button can be biased to the top (upper limit) of its travel range; 

this allows a greater range of button movement in the down direction and can eliminate or reduce 
a loss of force that may occur for negative altemation when the button limit is reached. A 
physical spring (e.g. a leaf spring or other type of spring) can be used to bias the button to the top 
of its travel. This may cause, in some embodiments, the button to stick up above the top surface 

20 . of the mouse housing and increased the finger force and stroke to close the button swdtch. 

This embodiment can altematively provide a button bias that is spring balanced and held 
in the center of its travel. Spring biasing the button tends to provide more effective force 
sensations to the user than without the spring biasing. 

Embodiments including haptic sensations on a mouse button may be more suitable for 
25 focused, high concentration tasks such as desktop applications. One advantage on other designs 
is its output of low frequency forces, allowing users to receive a good illusion of surface profile 
and texture as the cursor is moved across icons and menus. In gaming applications, pushing 
down on the button svirface may overpower the forces. This is may not be desirable for particular 
games, e.g. shooting games. Additionally, the user may lose the feedback sensations when the 
30 index finger is not in place on the button. In some embodiments, the moving surface can be 
enlarged, or a surroimding portion of housing can be caused to move around the button (instead 
of the button being provided with forces, as described in copending application 09/1 56,802. This 
may also alleviate the button closure interference / long stroke issue since a standard button can 
be used. 
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FIGURE 7 is a perspective view of another embodiment 320 of a tactile mouse of the 
present invention. The upper portion 322 of mouse 320 is shown, which is intended to mate with 
a bottom portion, e.g. a base similar to those shown with respect to Figs. 3a and 3b, or other type 
of base. Two mouse buttons 324 and 326 are shown from the underside of the cover portion 322. 

5 The cover portion 322 includes a movable surface portion 328 which can be moved 

relative to the cover portion 322 (or other remaining main portion of the housing). In the 
example shown, the movable portion 328 is positioned on the side of the mouse, where the user's 
thumb may contact the portion 328 diuing normal operation of the mouse. In this embodiment, 
the movable portion 328 may be moved in a direction approximately perpendicular to the side 

10 surface of the mouse (or other sxuface that immediately surrounds . the movable portion) and 
approximately parallel to the x-axis of the mouse planar workspace, as shown by arrow 330. The 
moveable portion 328 can be coupled to the cover portion 322 by a spring or hinge that allows 
the outward motion of arrow 330. For example, foam can be used to act as a biasing spring to 
center the moving surface in its degree of freedom; other types of springs can also be used. This 

15 bias forces the user's thumb outward, when the mouse is gripped normally. In the embodimmt 
shown, the movable portion 328 does not have button functionality such as a switch activated by 
pressing the portion 328, but altemate embodiments can include such frmctionality if desired. 

A linear voice coil 332, or other type of actuator providing linear motion, is coupled to 
the cover portion 322 (or other portion of the housing). For example, the voice coil 332 can be 

20 coupled to an extension 324 of the housing 322. The voice coil 332 includes a linearly-moving 
bobbin 334 that is directly coupled to the movable portion 328 so that the voice coil actuator 332 
directly moves the portion 328. The movable portion 328 also magnetically centers itself in its 
degree of freedom due to the magnetic characteristics of the voice coil 332. One example of a 
linear voice coil suitable for use in mouse 320 is described in copending application no. 

25 09/156,802. 

Since the forces on the user are output only parallel to only one axis of mouse movement, 
such as the x-axis, forces meant for the y-axis can also be output on the x-axis-moving portion 
328. The mapping from x-axis and y-axis to a single x-axis may present some perceptual 
challenges for the user. For example, position-based effects may make less sense to tiie user in 

30 this embodiment tlian in embodiments providing z-axis or both x- and y-axis forces, but still may 
be entertaining for the user. CUcks and pops are not directional and are well-suited to this 
embodiment. In some embodiments, a second moveable housing portion and dedicated voice 
coil actuator, similar to the thimib portion 328 and actuator 332, can be positioned to better map 
y-axis forces, e.g. such a second movable portion can be positioned on the front or back of the 

35 mouse housing and contact the user's fingers or palm. 

■ • 
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Other embodiments can also be provided. For exaniple, the entire cover portion, or a 
designated area of the cover portion^ may be moved in the z-direction against the user's pakn or 
fingers by a voice coil actuator or other type of actuator that directly moves the cover portion. 
The upper portion of the mouse housing can be flexibly coupled to the lower portion or base of 
the mouse so that the upper portion can be moved on the z-axis relative to the lower portion. 
Kinesthetic forces niay not be perceived as easily by the user as tactile (e.g. vibration) forces, but 
this can be remedied by increasing the travel distance of the moving housing portion. Examples 
of such an embodiment are described in greater detail in U.S. Patent No. 6,088,019, which is 
incorporated herein by reference in its entirety. 

This embodiment offers some advantages in that the user is always experiencing force 
sensations while operating the mouse since the entire upper cover portion is moved* Some users 
may not palm the mouse in use, but rather grasp the side edges of the mouse. To accommodate 
this, the cover portion can be extended to the side areas or side grip surfaces or ridges can be 
made more pronounced to enhance feedback from the gap area in this grasp mode. It may not be 
necessary in some embodiments to palm the mouse to receive compelling tactile feedback due to 
feeling vibrations caused by the moving housing. If only a smaller portion of the upper housing 
portion is movable, then the user can avoid holding down and overpowering the moving portion. 
For example, displacing an island of plastic sealed by a bellows can provide just as effective 
force feedback as displacing the whole upper housing portion. 

Furthermore, a gap formed by the spUt housing, between the upper and lower shells, 
creates a differentially displaced surface. Since the two portions of mouse housing are pinched- to 
provide movement, the user may contact the gap when operating the mouse. When the two 
halves of the housing pinch together or apart, the user receives proportional infomiation due to 
feeling the size of the gap changing. In other embodiments, a flexible material can be used to fill 
the gap or the differential information can be conveyed in other ways, such as putting tactile 
ridges on the upper and lower halves. 

Another tactile mouse embodiment provides force feedback on a mouse wheel, such as a 
wheel 206 shown with reference to Figs. 3a and 3b. A rotary actuator can provide rotational 
forces about the axis of rotation of the wheel. A surface providing good fiiction between the 
user's finger and the wheel is well suited to allow the user to feel the force sensations during 
control of the wheel. Many force feedback mouse wheel embodiments are described in U.S. 
Patent No. 6,128,006, which is incorporated herein by reference in its entirety. 

Merging any two or more features of the above embodiments into a single hybrid design 
can also be accomplished. Several of the functions and features can be combined to achieve a 
single design that, for example, has the mechanical simpUcity of the moving upper housing 
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design and the distinct focused or localized feedback of the haptic mouse button. Better hybrid 
designs incoiporate multiple implementations with reduced nimabers of actuators. For example, 
cost is much reduced if a single actuator can be used to output forces on the upper shell as well as 
a mouse button. 



Component Embodiments 

Any of the above embodiments for a haptic mouse can make use* of a variety of types of 
actuators. The lowest cost actuators providing reasonably hi^ performance are the. most 
desirable for the consumer market. For example, a small DC rotary motor provides good 
harmonic actuation with decent bandwidth from DC to about ISO Hz. There are also many types 
of models available. 

A solenoid can also be used. This actuator is not as desirable as the DC motor since it 
tends to deUver httle haptic value for the material and power expense; solenoids are typically not 
good at providing constant force over a useful stroke. Solenoids, however, may work well in 
some embodiments to generate a digital "pop" or pulse effect. An ofif^the-shelf solenoid can be 
biased to generate a quasi-linear force vs. stroke profile, and the traasmission may be simpler in 
those embodiments requiring linear motion since the solenoid abready provides linear motion. 

A shape memory alloy (SMA) wire with constant current drive circuit can also be used. 
This actuator is able to provide forces up to 100 Hz, especially "pops" in the range of 30 Hz. 
This can be a very forceful actuator; the operation of such a component is well known to those of 
skill in the art. 

A speaker or voice coil motor (VCM) can also be used. Off-the-shelf speakers are 
optimized to move a column of air. The return path and bobbin parts that can fit in a mouse 
housing volume may not produce enough force or have enough stroke to be useful. However, a 
custom voice coil can be designed to provide a useful stroke and high output force over that 
stroke. This actuator can operate sufficiently well and can be manufactured in high volume by 
leveraging off of an existing industry, such as the audio voice coil industry. 

For actuator couplings and transmissions, many components may be suitable. For 
example, a lead screw capable of being back driven can be used to couple a moving member to 
the actuator. The lead screw in some embodiments can incorporate a spring suspension to center 
the actuator. A molded flexure linkage driven with an eccentric cam moving in a slot can also be 
used. Alternatively, a one piece Uving hinge linkage (flexure) can be used to eliminate all pin 
joints and serve as the connection between the actuator and the housing. Examples of such 
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flexiires are described in copending application no. 09/585,741 and 60 / filed 9/28/00 and 

entitled "Device and Assembly for Providing Linear Inertial Sensations," both incorporated 
herein by reference in their entirety. 

User Interface Features 

FIGURE 8 is a diagram of display screen 26 of host computer 14 showing a graphical 
user interface, which is one type of computer-implemented graphical enviroiraient with which the 
user can interact using the device of the present invention. The haptic feedback mouse of the 
present invention can provide tactile sensations that make interaction v^th graphical objects more 
compelling and more intuitive. The user typically controls a ciirsor 400 to select and/or 
manipulate graphical objects and information in the graphical user interface. The cursor is 
moved according to a position control paradigm, where the position of the cursor corresponds to 
a position of ttie mouse in its planar (x-y) workspace. Windows 402, 404 and 406 display 
information from application programs running on the host computer 14. Menu elements 408 of 
a menu 410 can be selected by the user after a menu heading or button such as start button 41 1 is 
selected. Icons 412 and 414 and web link 416 are displayed features that can also be selected. 
Scroll bars, buttons^ and other standard GUI elements may also be provided. 

Tactile sensations associated with these graphical objects can be output using the 
actuator(s) of the device based on signals provided from the local microprocessor and/or host 
computer. A variety of haptic sensations that can be output on the housing and/or on a movable 
element of the device, and can be associated with GUI elements, including pulses, vibrations, 
textures, etc., are described in copending application nos. 09/456,887 and 09/504,201, 
incorporated herein by reference in their entirety. 

There are several desirable user interface . features for the mouse embodiments described 
herein. A high quality, crisp feeling to the sensations,' such as pulses or pops, on graphical 
objects such as scroll bars and menu items is appealing to users. Feeling a cUck or pop when 
entering or exiting an area on the GUI is helpful to locate the item haptically for the user. Tones, 
i.e. fixed magnitude variable frequency vibrations, can provide a fiill range of haptic sensations. 
High quaUty vibrations with varying magnitude aad frequency, and good low frequency periodic 
forcefiil displacements provide a variety of high-quality feels to graphical objects. Window 
boundaries can also be associated with a spring under the finger button, in appropriate 
embodiments. 
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Preferably, system events and sounds are mapped to haptic feedback sensations output by 
the mouse. Textures can also be implemented, e.g. x- and y-axis forces mapped to z-axis forces. 
Textures can, for example, distinguish window fields and areas or other areas of the graphical 
environment. Haptic feedback can also be output to the user to confirm the pressing of a key or a 
button by the user. When an icon or other object is dragged by the cursor, a sensation of icon 
weight can be implemented as a vibration "tone," where the tone firequency indicates weight of 
the selected object; for example, a low firequency vibration signifies a heavy or large graphical 
object or a large data size (e.g. in bytes) of a selected or dragged object, while a high fi*equency 
vibration indicates a small or lightweight object. To avoid disconcerting jarring effects as the 
cursor crosses icons, the force magnitude can be reduced (or otherwise adjusted) as a .fimction of 
cursor speed in the GUI. 

Mouse Button Sensations 

Additional user interface features can be provided for particular embodiments. For 
example, for the embodiment 300 or 270 providing haptic feedback on a button, several user 
interface haptic feedback sensations can be provided. Some compelling haptic sensations do not 
require a position sensor to determine a position of the button in its degree of fi^eedorn. 

For example, "soft spots'' or variable compliance surfaces can be provided on objects or 
areas in the GUI. When the user moves the cursor over a button, icon, menu item, or other 
selectable target (surface, object, or area), the pressing. force required by the. user to complete a 
button actuation is decreased noticeably by reducing resistance force in that direction of the 
button and/or providing an assistive force in that direction of button motion. This may give the 
user the perception of an active detent without using position-based forces to guide the mouse to 
the target. A vector force that doubles (or otherwise increjases) the stiffaess of the button can be 
used to require a greater pressing force to actuate the button when the cursor is not positioned 
over a selectable target or particular types or instances of selectable targets. 

If a sensor, such as a low-resolution encoder or potentiometer, is added to determine 
button position in its degree of freedom, additional sensations can be provided. For example, 
"piercing layers" can provide the user with the sensation of a third dimension into the plane of 
the screen. The graphical environment or application may have several windows or other objects 
which are "layered" based on when the window was opened and which windows have been made 
active "over" other windows. For example, window 404 is displayed on top of window 406, and 
window 402 is displayed on top of the windows 404 and 406. Typically, only one window is 
"active" at one time, e.g. accepts input from a keyboard or other input device; for example, the 

• ■ 
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5 

active window can have a differently-colored title bar 403 or other indicator. Jt can be 
convenient to toggle rapidly through such windows (or other types of layers). The haptic 
feedback mouse button of the present invention can provide this functionality by outputting a 
progressive spring force with detents overlaid on the spring. When in a layer selection mode, the 
5 moving of the button downward causes lower layers to become active, where distinct positions of 
the button can each be associated with a particular layer. A detent force or pulse output on the 
button can tactilely indicates when another layer is to be "punctured'' by the cursor and become 
active. 

• * 

For example, positioning the cursor over a blank spot in an active window 402 can put 

10 the mouse and cursor in a layer selection context or mode. The . user then presses the rnouse 
button until the cursor "pierces" through the current layer which causes a distinct puncture force 
effect such as a detent^ or jplt, and window 404 (or other object) at a new layer becomes active. 
Continuing to depress the button to a lower position will pierce yet another layer so that window 
406 becomes active, and so on, where each layer provides a puncture effect, such as a small 

IS resistance force (so that the user does not accidentally move the button into the next layer). 
When the iiser arrives at the desired window or layer, the button is released, which infomis 
firmware or software that a particular number of layers have been punctured and which window 
at a lower layer should be active and displayed on top. Puncturing successive layers can cause 
the successive windows to appear one after another as the active window. This feature can also 

20 be useful for application programs having several windows, like SolidWorks™. Such a feature 
would alleviate the use of keys or menus to toggle between, for example, part and assembly 
windows, which can be a distraction for the user. It can be much faster to pull the cursor to a 
blank area of the screen where puncturing and depressing functions let the user rapidly select the 
next window without doing any targeting at aU. This feature is also applicable to drawing 

25 programs, in which the user often organizes a drawing into different layers to allow the user to 
select, edit, and/or view only the parts of the drawing on a single layer at one time. A uiser can 
access the different drawing layers using the method described above. 

In some embodiments, if the user releases the button and then depresses the button again, 
the **puncture holes" the user previously made allow the button to be depressed more easily 

30 through those previously-punctured layers and are signaled by significantly diminished spring or 
detent forces or distinctiy different force profiles. The user knows which layer is enabled by how 
many decreased-force punctures the user feels before reaching an unpunctured layer, which has a 
noticeably higher force (a stiff mbber diaphragm is a good analogy). In some embodiments, 
double clicking on the impvmctxired layer causes the selected window to be displayed as the 

35 active layer. This example requires at least a crude position sensor, perhaps an encoder with 
several (e.g. about 64) coxmts over the stroke of the actuator. The value of such a feature would 
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depend on how well integrated the application is. Li one embodiment, an application program or 
GUI can determine how many windows are currently open and can spatially subdivide the button 
travel distance accordingly to allow constant spacing between puncture points. 

Another haptic sensation and user interface feature are layers with inertial or a ^turnstile." 
In such a layer implementation, a window or other graphical selected object can be considered to 
be "attached" to the mouse button, where moving the mouse button down moves the window 
"into" the screen to a different, lower layer. For example, when moving the cursor to a blank 
area of an active window 402, the user can depress the button and feel the inertia of the window 
402 and push that window into the background, behind other windows 404 and 406, so that the 
window 404 at the next highest level becomes active. As the next window 404 becomes active, 
the xiser feels a detent in the button's Z-axis signifying that the next window is now active. An 
analogy is a "turnstile" having multiple sections, where as each section becomes active, the user 
receives haptic feedback. This could also be used for spin boxes: Animations can show a 
window that has been "pushed" into the background as spirming into the screen and away. The 
inertial sensation can be a resistive force on the button and can be related to window size or other 
characteristics of the window. Again, a low-^resolution position sensor is desirable to sense the 
position of the button in its degree of freedom. 

Another button user interface feature of the present invention is a rate control button. The 
"layers" described above can be extended further by allowing tKat the same actuator and 
displaced surface and sensor assembly can be used to implement rate control at a surface 
function. For example, the cursor can be moved over a control such as a volume button. The 
user then moves the mouse button down to a first detent or pulse. The detent signifies that the 
volimie control is selected and that a rate control mode has been entered. The user then moves 
the mouse button up or down, and this controls the actual volmne level. For example, the 
volume can be adjusted a rate proportional to the distance of the button from its origin (centered) 
position. The rate control mode can be exited by, for example, allowing the button to move to its 
highest level, by pressing another button, etc. Preferably, a spring force resists the motion of the 
mouse button in rate control mode to allow greater control by the user. 

Rate control with an active buttpn can also be iiseful for scrolling documents or other 
objects. For example, pushing the button a greater distance dpwn (against a spring force) can 
increase the speed of scrolling, and allowing the button to move upward can decrease the 
scrolling speed, similar to the scrolling in the Wingman force feedback mouse from Logitech 
Corp. Since most scrolling is vertically oriented in the GUI, this is vvell correlated to a vertical 

• ■ 

button depression and is a natural feature. 
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Multiple switch actions can also be implemented using a haptic button. While 
conventional mouse buttons are fixed-movement mechanical buttons, the haptic feedback button 
with a position sensor of the present invention can become a huge variety of buttons with 
different force versus depression / actuation profiles implemented in software and using die 
5 actuator. Profiles such as a long stroke with very linear force or a short stroke with over-center 
snap action (toggle action) are possible with the same hardware. Other possibiUties include 
buttons that vibrate when the user begins to depress them and then warn the user more 
aggressively when the user has sUjghtly moved the button as if he or she is about to click the 
button. 

10 . Other button effects can be specially tailored for the embodiment 270 of Fig. 5, which 

uses a stop and a rotating eccentric mass to provide forces on the button. For example, 
rudimentary layer effects can be generated which do not involve rapid force reversals of the type 
felt piercing through a diaphragm, for instance. If the button is connected to a position sensor 
and the eccentric mass can be moved to bear against a stop anywhere in the movement range of 

IS the button, then kinesthetic forces (such as springs) can be output in one direction anywhere in 
that button's range of motion. CUcks (pulses) and pops can be generated by the inertial coupling 
of simple mass rotation, which can transmit a harmonic burst into the mouse housing for subtle 
pops. Alternatively, the mass can be controlled to rapidly engage a stop to generate a harsh 
knock or popping effect. 

20 Superposition of haptic effects can also be achieved with the embodiment 270. While the 

actuator is forcing the eccentric mass against a stop to provide a kinesthetic force on a movable 
surface (based on a DC drive signal), a high fi-equency harmonic signal may be applied to the 
actuator to output a vibration on the movable surface. This would allow the layers 
implementation above to include layers having different '"tones" (vibrations of different 

25 firequencies) when punctured; also, the tone can change firequency as the layer is moved, 
deformed, or manipulated. Preferably, the DC signal tliat forces the mass against the stop is 
always at least slightly greater in magnitude than the maximum negative alternation of the 
superimposed harmonic signal; this prevents the mass from moving off the stop (negative 
direction) and moving back into it and thus avoids a "chatter" of the mass. • 

30 Another control scheme can be provided for a rotating mass with slot and pin action built 

into the mouse button to manage cUcks and pops with more complex effects occurring 
simultaneously. Such a condBguration is shown in FIGURE 9, where the actuator 450 is coupled 
to a slot member 452 (mass) having a slot 454. A movable member 456, such as a button or 
portion of the housing, is coupled to a pin 458 that extends into the slot 454, The slot 454 is 

35 made wider than the pin, so that the actuator 450 can drive the slot member 452 harmonically 
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without contacting the pin 458 and provide inertial sensations to the housing. In addition, the 
slot member 452 can. engage the pin 458 to move the member 456 and provide kinesthetic forces 
on the member 456. The control scheme for superposition of forces would, first, slowly rotate 
the member 452 against gravity until the pin 458 engages the side of slot 454. This can be a 
default position so that the actuator is instantly able to respond to force commands without 
discontinuities. A high current is then commanded to produce a vertical force on the button 456. 
The current is maintained to maintain the slot member in the upward direction, and a harmonic 
signal is superimposed on the DC signal to oscillate the slot member and provide a vibration on 
the button in addition to the kinesthetic force; the DC signal prevents chatter of the slot member 
against the pin. . The current can be turned off to allow gravity to retum the slot membor to its 
neutral position. 

While this invention has been described in terms of several preferred embodiments, it is 
contemplated that alterations, permutations and equivalents thereof will become apparent to 
those skilled in the art upon a reading of the specification and study of the drawings. For 
example, the various embodiments disclosed herein can provide haptic sensations in a wide 
variety of types of interface devices, handheld or otherwise. Fiurthermore, certain terminology 
has been used for the purposes of descriptive clarity, and not to limit the present invention. It is 
therefore intended that the following appended claims include alterations, permutations, and 
equivalents as fall Avithin the true spirit and scope of the present invention. • . 
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